Apparatus and method

ABSTRACT

Apparatus including circuitry configured to: acquire correlation signals by correlating a photodetected signal with respective phase-shifted reference signals, the correlation signals comprising quadrature correlation signals and in-phase correlation signals. The circuitry is configured in such a way that the mean time of acquisition of the quadrature correlation signals and the mean time of acquisition of the in-phase correlation signals are substantially equal.

TECHNICAL FIELD

The present disclosure generally pertains to Time-of-Flight imagingapparatus and methods.

TECHNICAL BACKGROUND

Time-of-Flight (ToF) 3D imaging apparatus are known for providinginformation concerning the distance to an object by an analysis of theTime-of-Flight from a light source to the object and back. ToF 3Dimaging devices (or 3D ToF cameras) rely on the ToF principle whichrequires a plurality of subsequent measurements as from which depthcomputations are performed to estimate real distance of objects from acamera. Each measurement has its own integration time and the period oftime during which the plurality of measurements required to determine adepthmap also has a determined duration.

ToF cameras compute 3D images or depthmaps from sets of subsequentacquisitions/measurements taken according to the ToF principle (shown inFIG. 1 and explained hereafter). When objects within the scene moveduring the complete set of subsequent measurements (or if the cameramoves within the scene and with respect to the objects in a way makingthe distances change over the time), a negative impact on the computeddepth may occur as each subsequent measurement at each pixel will notcorrespond to the same part of the scene, and the computed depthmeasurement extracted from the said subsequent measurement according tothe ToF principle will provide inaccurate depth estimates (called motionblur in 2D imaging).

In order to determine the distance of objects from a camera, aphotodetected signal is usually correlated with four electricalreference signals that are shifted by 0°, 180°, 90° and 270°respectively, compared to the original optical signal. The acquisitionshould be adapted to minimize the error introduced in the depthcomputation for a scene wherein a dynamic aspect is involved (i.e.motion of the camera or of the scene/objects). The sequences ofcorrelation measurements in the state of the art have been set asfollows: a first set of a 0° and then of a 90° correlation measurementsis taken subsequently, and then a second set of a 180° and then of a270° correlation measurements is taken subsequently. In order todecrease the impact of camera movement or scene changes when the fourcorrelations are measured, one tried to reduce the errors caused bychanges in the scene (“motion blur”) by measuring the four correlationsas fast as possible, or one would use additional signal processing toidentify the regions in which motion blur had occurred.

Although there exist such techniques for decreasing the impact of cameramovement or scene changes, it is generally desirable to providesolutions that are more efficient in decreasing the impact of cameramovement or scene changes.

SUMMARY

According to a first aspect the disclosure provides an apparatuscomprising circuitry configured to: acquire correlation signals bycorrelating a photodetected signal with respective phase-shiftedreference signals, the correlation signals comprising quadraturecorrelation signals and in-phase correlation signals, wherein thecircuitry is configured in such a way that the mean time of acquisitionof the quadrature correlation signals and the mean time of acquisitionof the in-phase correlation signals is substantially equal.

According to a further aspect the disclosure provides an apparatuscomprising circuitry configured to: acquire correlation signals bycorrelating a photodetected signal with respective phase-shiftedreference signals, the correlation signals comprising a first set ofcorrelation signals and a second set of correlation signals, wherein thecircuitry is configured in such a way that the average phase of thefirst set of correlation signals and the average phase of the second setof correlation signals are orthogonal or substantially orthogonal toeach other, and wherein the circuitry is configured in such a way thatthe mean time of acquisition of the first set of correlation signals andthe mean time of acquisition of the second set of correlation signalsare equal or substantially equal.

According to a further aspect the disclosure provides a methodcomprising acquiring correlation signals by correlating a photodetectedsignal with respective phase-shifted reference signals, the correlationsignals comprising quadrature correlation signals and in-phasecorrelation signals, wherein the acquiring of the correlation signals isperformed in such a way that the mean time of acquisition of thequadrature correlation signals and the mean time of acquisition of thein-phase correlation signals is substantially equal.

According to a further aspect the disclosure provides a methodcomprising: acquiring correlation signals by correlating a photodetectedsignal with respective phase-shifted reference signals, the correlationsignals comprising a first set of correlation signals and a second setof correlation signals, wherein the average phase of the first set ofcorrelation signals and the average phase of the second set ofcorrelation signals are orthogonal or substantially orthogonal to eachother, and wherein the mean time of acquisition of the first set ofcorrelation signals and the mean time of acquisition of the second setof correlation signals are equal or substantially equal.

Further aspects are set forth in the dependent claims, the followingdescription and the drawings.

Before explaining embodiments of the disclosure in more detail withreference to the drawings, some general explanations are made.

Circuitry may be any kind of electronic circuitry, comprising integratedcircuitry such a microchips, a processing unit such as a CPU, LED and/orlaser driver electronics, or the like.

A photodetected signal may for example correspond to light that isobtained by a camera sensor of a ToF camera. For example, in ToFimaging, a photodetected signal may correspond to light that isreflected from a scene that is illuminated with modulated light.

The photodetected signal may be obtained by a ToF camera system thatresolves distance based on the known speed of light, measuring theTime-of-Flight of a modulated light signal between the camera and thesubject for each point of the image. Various technologies forTime-of-Flight cameras may be used in the context of the embodiments,for example RF-modulated light sources with phase detectors, or rangegated imagers. The reflected light may for example be captured by asensor which comprises a pixel array, where a single pixel consists of aphoto sensitive element (e.g. a photo diode) that converts the incominglight into a current.

In ToF imaging, a reference signal is for example correlated with aphotodetected signal to generate a correlation signal. The referencesignal may be an electric reference signal, or the like. According tosome embodiments, the reference signals are electric reference signalsthat are phase-shifted by 0°, 180°, 90° and 270° respectively, comparedto a modulated light signal.

Quadrature correlations may for example relate to reference signals thatare phase-shifted by 90° or substantially 90°, or 270° or substantially270°, whereas in-phase correlation may for example relate to referencesignals that are phase-shifted by 0° or 180° or substantially 0° orsubstantially 180°.

According to the embodiments, the mean time of acquisition of thequadrature correlation signals and the mean time of acquisition of thein-phase correlation signals is substantially equal. If the mean time ofacquisition of the quadrature correlation signals and the mean time ofacquisition of the in-phase correlation signals is substantially equalthis may result in that the mean time of acquisition of correlationsused in a numerator of an equation that describes the phase-shift anglebetween a modulated light signal and a photodetected signal, and themean time of acquisition of correlations in the denominator of theequation that describes this phase-shift angle is substantially equal.That is, if the mean time of acquisition of the quadrature correlationsignals and the mean time of acquisition of the in-phase correlationsignals is substantially equal, this may result in that the phase offseterror that results in motion blur is significantly reduced.

According to some embodiments, the correlation signals are quadraturemodulation signals and the circuitry is configured to acquire a firstand a last correlation signal on 180° opposing phase, and a second andthird correlation signal also on 180° opposing phase.

According to some embodiments, the first and a last correlation signalare quadrature correlation signals, whereas the second and thirdcorrelation signals are in-phase correlation signals.

According to yet other embodiments, the first and a last correlationsignal are in-phase correlation signals, whereas the second and thirdcorrelation signals are quadrature correlation signals.

According to some embodiments, the circuitry is configured to firstacquire subsequently a first set of 0° and then of 180° correlationmeasurements, and then acquire subsequently a second set of 90° and thenof 270° correlation measurements.

According to some embodiments, the circuitry is configured to: acquire a0° correlation signal at a first time T₀, acquire a 90° correlationsignal at a second time T₀+ΔT, acquire a 270° correlation signal at athird time T₀+2·ΔT, and acquire a 180° correlation signal at a last timeT₀+3·ΔT, where T₀ is a time when the acquisition of the correlationsignals starts, and where ΔT is a predefined time interval.

According to some embodiments, the circuitry is configured to acquirethe correlation signals according to alternative phase-shift sequences.

For example, all phases of the correlation signals may be shifted by thesame predefined phase angle without departing from the general conceptdescribed in this application.

That is, for example, the circuitry may also be configured to acquirecorrelation signals by correlating a photodetected signal withrespective phase-shifted reference signals, the correlation signalscomprising a first set of correlation signals and a second set ofcorrelation signals, wherein the circuitry is configured in such a waythat the average phase of the first set of correlation signals and theaverage phase of the second set of correlation signals are orthogonal orsubstantially orthogonal to each other, and wherein the circuitry isconfigured in such a way that the mean time of acquisition of the firstset of correlation signals and the mean time of acquisition of thesecond set of correlation signals are equal or substantially equal.

In particular, the first set of correlation signals may comprisequadrature modulation signals that are phase-shifted by a predefinedphase angle and the second set of correlation signals may comprisein-phase correlation signals that are phase-shifted by the samepredefined phase angle.

For example, the first set of correlation signals may comprisequadrature modulation signals and the second set of correlation signalmay comprise in-phase correlation signals that are phase-shifted by anarbitrary phase angle.

The apparatus may further comprise an illumination unit for illuminatinga scene with a modulated light signal. The illumination unit may forexample be a light emitting diode (LED), in particular a laser diode, orthe like.

The apparatus may further comprise an imaging sensor for receiving thephotodetected signal. The sensor may for example comprise a CCD pixelarray.

A method according to the embodiments may for example comprise acquiringcorrelation signals by correlating a photodetected signal withrespective phase-shifted reference signals, the correlation signalscomprising quadrature correlation signals and in-phase correlationsignals, wherein the acquiring the correlation signals is performed insuch a way that the mean time of acquisition of the quadraturecorrelation signals and the mean time of acquisition of the in-phasecorrelation signals are substantially equal.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are explained by way of example with respect to theaccompanying drawings, in which:

FIG. 1 schematically depicts the basic operational principle of a ToFcamera;

FIG. 2 provides an example of signals involved in a Time-of-Flightmeasurement;

FIG. 3 represents the photodetected signal S_(φ) in its polar form;

FIG. 4 shows an example of a correlation measurement control sequence;

FIG. 5 shows a ToF 3D example image provided by the state of the artphase-shift sequence and that shows substantial motion blur;

FIG. 6 schematically shows an example of how the mean time ofacquisition is determined for the numerator Q and denominator I;

FIG. 7 schematically shows a method of acquiring a ToF image using aphase-shift sequence according to an embodiment of the disclosure; and

FIG. 8 shows a rear view of a hand with a pointing index finger, thehand performing a circular gesture, with the scene being acquired by a3D ToF camera located on the top of the hand.

DETAILED DESCRIPTION OF EMBODIMENTS

FIG. 1 schematically illustrates the basic operational principle of aToF camera 3. The ToF camera 3 captures 3D images of a scene 15 byanalyzing the Time-of-Flight of light from a dedicated illumination unit18 to an object. The ToF camera 3 includes a camera, for instance a 3Dsensor 1 and a processor 4. A scene 15 is actively illuminated with amodulated light 16 at a predetermined wavelength using the dedicatedillumination unit 18, for instance with some light pulses of at leastone predetermined frequency generated by a timing generator 5. Themodulated light 16 is reflected back from objects within the scene 15. Alens 2 collects the reflected light 17 and forms an image of the objectsonto the imaging sensor 1 of the camera. Depending on the distance ofobjects from the camera, a delay is experienced between the emission ofthe modulated light 16, e.g. the so-called light pulses, and thereception at the camera of those reflected light pulses 17. Distancesbetween reflecting objects and the camera may be determined as functionof the time delay observed and the speed of light constant value.

The distance of objects from the camera can be calculated as follows.

FIG. 2 shows an example of signals involved in a Time-of-Flightmeasurement. A modulation signal S (16 in FIG. 1) is sent towards anobject. After reflection on the object, a signal S_(φ) is detected by aphotodetector that receives reflected light (17 in FIG. 1). This signalS_(φ) is phase-shifted by a phase p compared to the original signal S,due to the travelling time. For instance, if the signal S is asinusoidal wave of the form:

S=A cos(2πft)  (eq. 1)

then, S_(φ) can be seen as a phase-shifted wave with the followingmathematical form:

S _(φ) =A cos(2πft+φ)=A cos(2πft)cos(φ)−A sin(2πft)sin(φ).  (eq. 2)

By defining the so-called in-phase I and quadrature Q components by:

I=A cos(φ) and Q=A sin(φ)  (eq. 3, 4)

then S_(φ) can be written as

S _(φ) =I cos(2πft)−Q sin(2πft).  (eq. 5)

This equation enables representing S_(φ) in its polar form, as a vector,represented in FIG. 3, with φ being the phase of S_(φ) and r being aparameter corresponding to the amplitude A of the signal S_(φ) and beingalso related to the so-called confidence. φ, I and Q are key parametersfor measuring the distance of objects from a camera. To measure theseparameters, the photodetected signal S_(φ) is usually correlated withelectrical reference signals named S_(I), S_(Ī), S_(Q) and S _(Q) .S_(I), S_(Ī), S_(Q) and S _(Q) are electrical reference signals shiftedby 0°, 180°, 90° and 270° respectively, compared to the original opticalsignal S, as illustrated in FIG. 2. The correlation signals obtained aredefined as follows:

S _(φ,I) =S _(φ) ·S _(I)

S _(φ,Ī) =S _(φ) ·S _(Ī)

S _(φ,Q) =S _(φ) ·S _(Q)

S _(φ,Q) =S _(φ) ·S _(Q) .  (eq. 6-9)

Then, the two parameters I and Q can be calculated such that:

I=A _(S)·α·(S _(φ,I) −S _(φ,Ī)) and

Q=A _(S)·α·(S _(φ,Q) −S _(φ,Q) ).  (eq. 10-11)

A_(S) and α are, respectively, the amplitude change of the photodetectedsignal S_(φ) and the efficiency of the correlation, which both aremeasured during operation.

The extraction of φ depends on the shape of the modulation signal S. Forexample, if S is a sine wave, then

$\begin{matrix}{\phi = \left\{ \begin{matrix}{\arctan \frac{Q}{I}} & {{{if}\mspace{14mu} I},{Q \geq 0}} \\{{\arctan \frac{Q}{I}} + \pi} & {{{if}\mspace{14mu} I} < 0} \\{{\arctan \frac{Q}{I}} + {2\; \pi}} & {{{{if}\mspace{14mu} Q} < 0},{I \geq 0}}\end{matrix} \right.} & \left( {{{eq}.\; 12}\text{-}14} \right)\end{matrix}$

Once the phase φ is known, the distance D_(φ) of objects from a cameracan be retrieved according to the following formula:

$\begin{matrix}{D_{\phi} = \frac{c \cdot \left( {\phi + {2\; {\pi \cdot n}}} \right)}{4\; {\pi \cdot f_{mod}}}} & \left( {{eq}.\mspace{14mu} 15} \right)\end{matrix}$

where c is the speed of light, f_(mod) is the modulation frequency and nis an integer number which relates to phase ambiguity.

Scene or Camera Changes

The arctan term of equations (12-14) relies on the assumption thatneither the scene nor the camera has changed when the four correlationswere measured. For instance, in the case that φ is constant, and A_(S)increases by a constant ΔA_(S) between correlations taken with 0°, 90°,180° and 270° signal shifts, the phase offset error is:

$\begin{matrix}{{\Delta \; \phi} = {{\arctan \left( \frac{{\left( {I + {4\; \Delta \; A_{S}}} \right) \cdot \sin}\; \phi}{{\left( {I + {2\; \Delta \; A_{S}}} \right) \cdot \cos}\; \phi} \right)} - \phi}} & \left( {{eq}.\mspace{14mu} 16} \right)\end{matrix}$

More precisely, the sequences of correlation measurements in the stateof the art have logically been set as follows: A first set of a 0°(S_(φ,I)) and then of a 90° (S_(φ,Q) ) correlation measurements is takensubsequently, and then a second set of a 180° (S_(φ,Ī)) and then of a270° (S_(φ,Q) ) correlation measurements is taken subsequently. This isdue to ease of linear increase of the offset to go from one phase to thenext one; the increase being set according to a single and same offsetof 90° each time (requiring one single memory allocated to store theoffset).

FIG. 4 shows an example of a correlation measurement control sequence.The control sequence comprises an uFrame uF which comprises foursub-frames SF1, SF2, SF3, and SF4. Each sub-frame relates to a specificcorrelation measurement with respect to a specific reference signal.According to the state of the art phase-shift sequence, sub-frame SF1controls a 0° correlation measurement (S_(φ,I)), sub-frame SF2 controlsa 90° correlation measurement (S_(φ,Q)), sub-frame SF3 controls a 180°correlation measurement (S_(φ,Ī)), and sub-frame SF4 controls a 270°correlation measurement (S_(φ,Q) ).

Each sub-frame comprises an integration sequence, during which thecorrelation signal is received and integrated. The integration sequenceis followed by a R/O sequence, during which the integration result isread out. An R/O sequence is followed by an SF DT sequence, by which thesub-frame ends. DT is a Sub Frame Down Time, during which nothinghappens.

FIG. 5 shows a ToF 3D example image provided by the state of the artphase-shift sequence. As this ToF 3D image has been obtained when a usermoves the hand before the ToF 3D camera, the ToF 3D image showssignificant motion blur.

The embodiments described below relate to an advantageous arrangement ofthe acquisition over the time, said arrangement enabling theminimization of depth computation errors when estimated as from the ToFprinciple. They more precisely relate to reordering the signal shifts sothat the phase error is reduced.

The arrangement is such that the mean time of acquisition ofcorrelations used in the numerator (in eqs. 12-14 that describe thephase-shift angle φ) and the mean time of acquisition of correlations inthe denominator (in the eqs. 12-14 that describe the phase-shift angleφ) are equal.

For quadrature modulations like the ones described, this means that thefirst and last 3D pictures are taken on 180° opposing phase, and thesecond and third ones also, for instance in the improved phase-shifthereunder.

FIG. 6 schematically shows an example of how the mean time ofacquisition is determined for the numerator Q and denominator I in eqs.12-14 when using the phase-shift sequence 0° (S_(φ,I)), 90° (S_(φ,Q)),270° (S_(φ,Q) ), 180° (S_(φ,Ī)) of an embodiment of the invention. Thelower graph of FIG. 6 shows the acquisition of the correlation signalsS_(φ,Q) and S_(φ,Q) (quadrature correlations) for the numerator Q ineqs. 12-14 as signal amplitude A over time t. Correspondingly, the uppergraph of FIG. 6 shows the acquisition of the correlation signals S_(φ,Q)and S_(φ,Q) (in-phase correlations) for the denominator I in eqs. 12-14as signal amplitude A over time t. The acquisition of correlation signalS_(φ,I) starts at time T₀ and lasts for an integration time δt, theacquisition of correlation signal S_(φ,Q) starts at time T₀+ΔT and lastsfor an integration time δt, the acquisition of correlation signalS_(φ,Q) starts at time T₀+2·ΔT and lasts for an integration time δt, andthe acquisition of correlation signal S_(φ,Ī) starts at time T₀+3·ΔT andlasts for an integration time δt. It is assumed here for the sake ofexample that the integration time δt and the signal amplitude A(t)=A₀ isthe same for each correlation measurement. The mean time of acquisitiony_(Q) for the nominator correlations S_(φ,Q) and S_(φ,Q) thus is

$y_{Q} = {\frac{\int{{A(t)} \cdot t \cdot {dt}}}{\int{{A(t)} \cdot {dt}}} = {T_{0} + {\text{1/2}\Delta \; T} + {\text{1/2}\delta \; {t.}}}}$

Accordingly, the mean time of acquisition y_(I) for the denominatorcorrelations S_(φ,I) and S_(φ,Ī) is

$y_{I} = {\frac{\int{{A(t)} \cdot t \cdot {dt}}}{\int{{A(t)} \cdot {dt}}} = {T_{0} + {\text{1/2}\Delta \; T} + {\text{1/2}\delta \; {t.}}}}$

That is, with the phase-shift sequence of the example of FIG. 6, themean time of acquisition y_(Q) for the nominator correlations S_(φ,Q)and S_(φ,Q) (quadrature correlations) and the mean time of acquisitiony, for the denominator correlations S_(φ,I) and S_(φ,Ī) (in-phasecorrelations) is the same.

As in the embodiment of FIG. 6 it is assumed that the integration timeδt and the signal amplitude A(t)=A₀ is the same for each correlationmeasurement, the mean time of acquisition y_(Q) for the nominatorcorrelations (quadrature correlations) and the mean time of acquisitiony_(I) for the denominator correlations (in-phase correlations) areexactly the same. In other practical examples, the amplitudes A, thetime intervals ΔT and the integration period δt must not be identicalfor each correlation measurement. In such cases the mean time ofacquisition y_(Q) of the nominator correlations (quadraturecorrelations) and the mean time of acquisition y_(I) for the denominatorcorrelations (in-phase correlations) must not be exactly the same. It issufficient that they are substantially the same.

The above-described phase-shift sequence of the first embodiment ascompared with the phase-shift sequence of the state of the art isdisplayed in the following table.

Phase-shift Phase-shift sequence of sequence accord- the State of ing tofirst Picture Time I the Art embodiment 0 T₀ A_(S)  0°  0° 1 T₀ + ΔTA_(S) + ΔA_(S)  90°  90° 2 T₀ + 2 · ΔT A_(S) + 2 · Δ A_(S) 180° 270° 3T₀ + 3 · ΔT A_(S) + 3 · Δ A_(S) 270° 180°

This order, for instance, transforms equation 16 into equation 17 shownhereafter, which shows the error on the phase has been reduced to 0.

$\begin{matrix}{{\Delta \; \phi} = {{{\arctan \left( \frac{{\left( {I + {3\; \Delta \; A_{S}}} \right) \cdot \sin}\; \phi}{{\left( {I + {3\Delta \; A_{S}}} \right) \cdot \cos}\; \phi} \right)} - \phi} = 0}} & \left( {{eq}.\mspace{14mu} 17} \right)\end{matrix}$

Further phase-shift sequences of other embodiments are displayed in thetable below, where phase-shift sequence A is the same as the onedescribed in the table above:

Phase-shift sequence Picture A B C D E F G H 0  0°  0° 180° 180°  90° 90° 270° 270° 1  90° 270°  90° 270°  0° 180°  0° 180° 2 270°  90° 270° 90° 180°  0° 180°  0° 3 180° 180°  0°  0° 270° 270°  90°  90°

According to the phase-shift sequences above, a first and a lastcorrelation signal are acquired on 180° opposing phase, and a second andthird correlation signal are also acquired on 180° opposing phase.

According to yet other embodiments the correlation signals must notnecessarily be in-phase or quadratic to the emitted signal. For example,all phases of the correlation signals can be shifted by the same angle δso that the difference between them remains the same, and the beneficialeffects of the embodiments are still obtained. For example, instead ofthe phase-shift sequence A: 0°, 90°, 270°, 180°, the phase-shiftsequence A′: 0°+δ, 90°+δ, 270°+δ, 180°+δ can be used without departingfrom the gist of the embodiments.

According to these embodiments, the average phase of the phase-shiftedquadrature correlation signals and the average phase of thephase-shifted in-phase correlation signals are orthogonal to each other.For example, according to phase-shift sequence A′ above, the averagevalue of the shifted in-phase correlation signals 0°+δ, 180°+δ is 90°+δand the average value of the shifted quadrature correlation signals90°+δ, 270°+δ is 180°+δ. That is, the average value 90°+δ of the shiftedin-phase correlation signals and the average value 180°+δ of the shiftedquadrature correlation signals are orthogonal to each other(|(180°+δ)−(90°+δ)|=90°).

According to such embodiments, where all phases of the correlationsignals are shifted by the same angle δ, equation (16) takes thefollowing form:

$\begin{matrix}{{\Delta \; \phi} = {{\arctan \left( \frac{{\left( {I + {4\; \Delta \; A_{S}}} \right) \cdot \sin}\; \left( {\phi - \delta} \right)}{{\left( {I + {2\; \Delta \; A_{S}}} \right) \cdot \cos}\; \left( {\phi - \delta} \right)} \right)} - \left( {\phi - \delta} \right)}} & \left( {{eq}.\mspace{14mu} 16^{\prime}} \right)\end{matrix}$

Phase-shift sequences of yet other embodiments are displayed in thetable below:

Phase-shift sequence Picture I J K L M N O P 0  0°  0° 120° 120°  90° 90° 210° 210° 1  90° 210°  90° 210°  0° 120°  0° 120° 2 210°  90° 210° 90° 120°  0° 120°  0° 3 120° 120°  0°  0° 210° 210°  90°  90°

Again, all angles can be shifted by the same angle δ so that thedifference between them remains the same, and the beneficial effects ofthe embodiments are still obtained. For example, instead of thephase-shift sequence I 0°, 90°, 210°, 120°, the phase-shift sequence I′:0°+δ, 90°+δ, 210°+δ, 120°+δ can be used without departing from the gistof the embodiments. According to these embodiments, again, the averagephase of the phase-shifted quadrature correlation signals and theaverage phase of the phase-shifted in-phase correlation signals areorthogonal to each other. For example, according to phase-shift sequenceI′ above, the average value of the shifted in-phase correlation signals0°+δ, 180°+δ is 90°+δ and the average value of the shifted quadraturecorrelation signals 90°+δ, 270°+δ is 180°+δ. That is, the average value90°+δ of the shifted in-phase correlation signals and the average value180°+δ of the shifted quadrature correlation signals are orthogonal toeach other (|(180°+δ)−(90°+δ)=90°).

According to the phase-shift sequences above, a first and a lastcorrelation signal are acquired on 120° opposing phase, and a second andthird correlation signal are also acquired on 120° opposing phase.

According to such embodiments equation (16) takes the following form(which is the same as in eq. 16′):

$\begin{matrix}{{\Delta \; \phi} = {{\arctan \left( \frac{{\left( {I + {4\; \Delta \; A_{S}}} \right) \cdot \sin}\; \left( {\phi - \delta} \right)}{{\left( {I + {2\; \Delta \; A_{S}}} \right) \cdot \cos}\; \left( {\phi - \delta} \right)} \right)} - \left( {\phi - \delta} \right)}} & \left( {{eq}.\mspace{14mu} 16^{''}} \right)\end{matrix}$

In the prior art, one would reduce the errors caused by changes in thescene (“motion blur”) by measuring the four correlations as fast aspossible to reduce the impact of scene changes or one would useadditional signal processing to identify the regions in which motionblur had occurred. The embodiments described above, however, limit themotion blur which induces depth measurement errors in key common casesand reduces it significantly otherwise, without additional computation,and without a modification of the system, but with only modification ofthe method parameters.

FIG. 7 schematically shows a method of acquiring a ToF image using aphase-shift sequence according to an embodiment of the disclosure. At801, a correlation signal S_(φ,Q) is acquired by correlating aphotodetected signal S_(φ) with a respective phase-shifted referencesignal S_(Q). At 802, a correlation signal S_(φ,I) is acquired bycorrelating a photodetected signal S_(φ) with a respective phase-shiftedreference signal S_(φ). At 803, a correlation signal S_(φ,Ī) is acquiredby correlating a photodetected signal S_(φ) with a respectivephase-shifted reference signal S_(Ī). At 804, a correlation signalsS_(φ,Q) is acquired by correlating a photodetected signal S_(φ) with arespective phase-shifted reference signal S _(Q) . The correlationsignals S_(φ,Q), S_(φ,Q) , S_(φ,I), S_(φ,Ī) comprise quadraturecorrelation signals S_(φ,Q), S_(φ,Q) and in-phase correlation signalsS_(φ,I), S_(φ,Ī). The acquiring the correlation signals S_(φ,Q), S_(φ,Q), S_(φ,I), S_(φ,Ī) is performed in such a way that the mean time ofacquisition of the quadrature correlation signals S_(φ,Q), S_(φ,Q) andthe mean time of acquisition of the in-phase correlation signalsS_(φ,I), S_(φ,Ī) are substantially equal.

Effect of the Embodiments

FIG. 8 shows a rear view of a hand with a pointing index finger, thehand performing a circular gesture, with the scene being acquired by a3D ToF camera located on the top of the hand. The phase difference errorcomputed from the subsequently acquired correlations generates anerroneous depth measurement toward and away from the camera on theY-axis. The depth measurement error increases with the gradient betweena considered pixel and its surrounding pixels. The left image (FIG. 8a )has been obtained with the state of the art phase-shift sequence, andthe right image (FIG. 8b ) has been obtained with the phase-shiftsequence of an embodiment as described above. As can be seen in FIG. 8the motion blur according to the phase-shift sequence of the embodimentis reduced as compared to the motion blur according to the phase-shiftsequence of the prior art.

The embodiments apply to all situations wherein a ToF camera system isto be operated, and in particular wherein motion blur should beminimized and/or system constraints can be relaxed at constant motionblur e.g. moving hand/finger tracking e.g. in a car, moving full bodytracking, or 3D object scanning with moving object/camera.

All units and entities described in this specification and claimed inthe appended claims can, if not stated otherwise, be implemented asintegrated circuit logic, for example on a chip, and functionalityprovided by such units and entities can, if not stated otherwise, beimplemented by software.

In so far as the embodiments of the disclosure described above areimplemented, at least in part, using a software-controlled dataprocessing apparatus, it will be appreciated that a computer programproviding such software control and a transmission, storage or othermedium by which such a computer program is provided are envisaged asaspects of the present disclosure. That is, the methods as describedherein may also be implemented in some embodiments as a computer programcausing a computer and/or a processor to perform the method, when beingcarried out on the computer and/or processor. In some embodiments, alsoa non-transitory computer-readable recording medium is provided thatstores therein a computer program product, which, when executed by aprocessor, such as the processor described above, causes the methodsdescribed herein to be performed.

In some of the embodiments described above, quadrature modulatedcorrelation signals with four different correlation signals S_(φ,Q),S_(φ,Q) , S_(φ,I), S_(φ,Ī) are used. The embodiments are, however, notrestricted to this specific configuration. For example, in alternativeembodiments, more or less than four correlation signals could be used.

Likewise, in some of the embodiments described above, correlationsignals S_(φ,Q), S_(φ,Q) , S_(φ,I), S_(φ,Ī) are obtained by correlatinga photodetected signal S_(φ) with respective phase-shifted referencesignals S_(I), S_(Ī), S_(Q) and S _(Q) that are phase-shifted by 0°,180°, 90°, and 270°, respectively, compared to a modulated light signalS. The embodiments are, however, not restricted to this specificconfiguration. For example, in alternative embodiments, otherphase-shifts such as 0°, 90°, 120°, and 270°, or other combinations maybe used.

The skilled person will readily appreciate that the disclosure is notlimited to the specific processing orders described in the embodiments.For example processing 801 and example 802 in FIG. 7 can be exchanged.

It should be noted that the present technology can also be configured asdescribed below.

(1) An apparatus comprising circuitry configured to:

-   -   acquire correlation signals (S_(φ,Q), S_(φ,Q) , S_(φ,I),        S_(φ,Ī)) by correlating a photodetected signal (S_(φ)) with        respective phase-shifted reference signals (S_(Q), S _(Q) ,        S_(I), S_(Ī)), the correlation signals (S_(φ,Q), S_(φ,Q) ,        S_(φ,I), S_(φ,Ī)) comprising quadrature correlation signals        (S_(φ,Q), S_(φ,Q) ) and in-phase correlation signals (S_(φ,I),        S_(φ,Ī)),    -   wherein the circuitry is configured in such a way that the mean        time of acquisition of the quadrature correlation signals        (S_(φ,Q), S_(φ,Q) ) and the mean time of acquisition of the        in-phase correlation signals (S_(φ,I), S_(φ,Ī)) are equal or        substantially equal.        (2) The apparatus of (1), wherein the processor is configured to        acquire the correlation signals (S_(φ,Q), S_(φ,Q) , S_(φ,I),        S_(φ,Ī)) according to any one of the phase-shift sequences        (A-H):

Phase-shift sequence A B C D E F G H S_(φ,Q) S_(φ,) _(Q) S_(φ,) _(Q)S_(φ,) _(Q) S_(φ,I) S_(φ,I) S_(φ,) _(I) S_(φ,) _(I) S_(φ,I) S_(φ,) _(I)S_(φ,I) S_(φ,) _(I) S_(φ,Q) S_(φ,) _(Q) S_(φ,Q) S_(φ,) _(Q) S_(φ,) _(I)S_(φ,I) S_(φ,) _(I) S_(φ,I) S_(φ,) _(Q) S_(φ,Q) S_(φ,) _(Q) S_(φ,Q)S_(φ,) _(Q) S_(φ,) _(Q) S_(φ,Q) S_(φ,Q) S_(φ,) _(I) S_(φ,) _(I) S_(φ,I)S_(φ,I)(3) The apparatus of (1) or (2), wherein the reference signals (S_(I),S_(Q), S _(Q) , S_(Ī)) are phase-shifted by 0°, 90°, 270° and 180°respectively, compared to a modulated light signal (S).(4) The apparatus of anyone of (1) to (3), wherein the correlationsignals are quadrature modulation signals and the circuitry isconfigured to acquire a first and a last correlation signal on 180°opposing phase, and a second and third correlation signal also on 180°opposing phase.(5) The apparatus of anyone of (1) to (3), wherein the circuitry isconfigured to first acquire subsequently a first set of 0° and then of90° correlation measurements, and then acquire subsequently a second setof 210° and then of 120° correlation measurements.(6) The apparatus of (3), wherein the circuitry is configured to:

-   -   acquire the 0° correlation signal (S_(φ,I)) at a first time T₀,    -   acquire the 90° correlation signal (S_(φ,Q)) at a second time        T₀+ΔT,    -   acquire the 270° correlation signal (S_(φ,Q) ) at a third time        T₀+2·ΔT, and    -   acquire the 180° correlation signal (S_(φ,Ī)) at a last time        T₀+3·ΔT, where T₀ is a time when the acquisition of the        correlation signals (S_(φ,Q), S_(φ,Q) , S_(φ,I), S_(φ,Ī))        starts, and where ΔT is a predefined time interval.        (7) The apparatus of anyone of (1) to (4), wherein the processor        is configured to acquire the correlation signals (S_(φ,Q),        S_(φ,Q) , S_(φ,I), S_(φ,Ī)) according to any one of the        phase-shift sequences (A-H):

Phase-shift sequence Picture A B C D E F G H 0  0°  0° 180° 180°  90° 90° 270° 270° 1  90° 270°  90° 270°  0° 180°  0° 180° 2 270°  90° 270° 90° 180°  0° 180°  0° 3 180° 180°  0°  0° 270° 270°  90°  90°(8) The apparatus of anyone of (1) to (3), wherein the processor isconfigured to acquire the correlation signals (S_(φ,Q), S_(φ,Q) ,S_(φ,I), S_(φ,Ī)) according to any one of the phase-shift sequences(I-P):

Phase-shift sequence Picture I J K L M N O P 0  0°  0° 120° 120°  90° 90° 210° 210° 1  90° 210°  90° 210°  0° 120°  0° 120° 2 210°  90° 210° 90° 120°  0° 120°  0° 3 120° 120°  0°  0° 210° 210°  90°  90°(9) The apparatus of anyone of (1) to (3), further comprising anillumination unit (18) configured to illuminate a scene (24) with amodulated light signal (S).(10) The apparatus of (1) to (9), further comprising an imaging sensor(1) configured to receive the photodetected signal (S_(φ)).(11) An apparatus comprising circuitry configured to:

-   -   acquire correlation signals (S_(φ,Q), S_(φ,Q) , S_(φ,I),        S_(φ,Ī)) by correlating a photodetected signal (S_(φ)) with        respective phase-shifted reference signals (S_(Q), S _(Q) ,        S_(I), S_(Ī)), the correlation signals (S_(φ,Q), S_(φ,Q) ,        S_(φ,I), S_(φ,I)) comprising a first set of correlation signals        (S_(φ,Q), S_(φ,Q) ) and a second set of correlation signals        (S_(φ,I), S_(φ,Ī)),    -   wherein the circuitry is configured in such a way that the        average phase of the first set of correlation signals (S_(φ,Q),        S_(φ,Q) ) and the average phase of the second set of correlation        signals (S_(φ,I), S_(φ,Ī)) are orthogonal or substantially        orthogonal to each other, and    -   wherein the circuitry is configured in such a way that the mean        time of acquisition of the first set of correlation signals        (S_(φ,Q), S_(φ,Q) ) and the mean time of acquisition of the        second set of correlation signals (S_(φ,I), S_(φ,Ī)) are equal        or substantially equal.        (12) The apparatus of (11), wherein the first set of correlation        signals (S_(φ,Q), S_(φ,Q) ) comprises quadrature modulation        signals and the second set of correlation signal comprises        in-phase correlation signals (S_(φ,I), S_(φ,Ī)).        (13) The apparatus of (11) or (12), wherein the first set of        correlation signals (S_(φ,Q), S_(φ,Q) ) comprises quadrature        modulation signals that are phase-shifted by a predefined phase        angle and the second set of correlation signals comprises        in-phase correlation signals (S_(φ,I), S_(φ,Ī)) that are        phase-shifted by the predefined phase angle.        (14) The apparatus of anyone of (11) to (13), wherein the        processor is configured to acquire the first set of correlation        signals (S_(φ,Q), S_(φ,Q) ) and the second set of correlation        signals (S_(φ,I), S_(φ,Ī)) according to any one of the        phase-shift sequences (A-H):

Phase-shift sequence Picture A B C D E F G H 0  0°  0° 180° 180°  90° 90° 270° 270° 1  90° 270°  90° 270°  0° 180°  0° 180° 2 270°  90° 270° 90° 180°  0° 180°  0° 3 180° 180°  0°  0° 270° 270°  90°  90°(15) The apparatus of anyone of (11) to (13), wherein the processor isconfigured to acquire the first set of correlation signals (S_(φ,Q),S_(φ,Q) ) and the second set of correlation signals (S_(φ,I), S_(φ,Ī))according to any one of the phase-shift sequences (I-P):

Phase-shift sequence Picture I J K L M N O P 0  0°  0° 120° 120°  90° 90° 210° 210° 1  90° 210°  90° 210°  0° 120°  0° 120° 2 210°  90° 210° 90° 120°  0° 120°  0° 3 120° 120°  0°  0° 210° 210°  90°  90°(16). A method comprising:

-   -   acquiring correlation signals (S_(φ,Q), S_(φ,Q) , S_(φ,I),        S_(φ,Ī)) by correlating a photodetected signal (S_(φ)) with        respective phase-shifted reference signals (S_(Q), S _(Q) ,        S_(I), S_(Ī)), the correlation signals (S_(φ,Q), S_(φ,Q) ,        S_(φ,I), S_(φ,Ī)) comprising quadrature correlation signals        (S_(φ,Q), S_(φ,Q) ) and in-phase correlation signals (S_(φ,I),        S_(φ,Ī)),    -   wherein the acquiring the correlation signals (S_(φ,Q), S_(φ,Q)        , S_(φ,I), S_(φ,Ī)) is performed in such a way that the mean        time of acquisition of the quadrature correlation signals        (S_(φ,Q), S_(φ,Q) ) and the mean time of acquisition of the        in-phase correlation signals (S_(φ,I), S_(φ,Ī)) are        substantially equal.        (17) A computer program comprising instructions which, when        carried out on a processor, cause the processor to perform the        method of (16).        (18) A method comprising:    -   acquiring correlation signals (S_(φ,Q), S_(φ,Q) , S_(φ,I),        S_(φ,Ī)) by correlating a photodetected signal (S_(φ)) with        respective phase-shifted reference signals (S_(Q), S _(Q) ,        S_(I), S_(Ī)), the correlation signals (S_(φ,Q), S_(φ,Q) ,        S_(φ,I), S_(φ,Ī)) comprising a first set of correlation signals        (S_(φ,Q), S_(φ,Q) ) and a second set of correlation signals        (S_(φ,I), S_(φ,I)),    -   wherein the average phase of the first set of correlation        signals (S_(φ,Q), S_(φ,Q) ) and the average phase of the second        set of correlation signals (S_(φ,I), S_(φ,Ī)) are orthogonal or        substantially orthogonal to each other, and    -   wherein the mean time of acquisition of the first set of        correlation signals (S_(φ,Q), S_(φ,Q) ) and the mean time of        acquisition of the second set of correlation signals (S_(φ,I),        S_(φ,Ī)) are equal or substantially equal.        (19) A computer program comprising instructions which, when        carried out on a processor, cause the processor to perform the        method of (18).        (20) A computer program comprising instructions which, when        carried out on a processor, cause the processor to perform:    -   acquiring correlation signals (S_(φ,Q), S_(φ,Q) , S_(φ,I),        S_(φ,Ī)) by correlating a photodetected signal (S_(φ)) with        respective phase-shifted reference signals (S_(Q), S _(Q) ,        S_(I), S_(Ī)), the correlation signals (S_(φ,Q), S_(φ,Q) ,        S_(φ,I), S_(φ,Ī)) comprising quadrature correlation signals        (S_(φ,Q), S_(φ,Q) ) and in-phase correlation signals (S_(φ,I),        S_(φ,Ī)),    -   wherein the acquiring the correlation signals (S_(φ,Q), S_(φ,Q)        , S_(φ,I), S_(φ,Ī)) is performed in such a way that the mean        time of acquisition of the quadrature correlation signals        (S_(φ,Q), S_(φ,Q) ) and the mean time of acquisition of the        in-phase correlation signals (S_(φ,I), S_(φ,Ī)) are        substantially equal.        (21) A computer program comprising instructions which, when        carried out on a processor, cause the processor to perform:    -   acquiring correlation signals (S_(φ,Q), S_(φ,Q) , S_(φ,I),        S_(φ,Ī)) by correlating a photodetected signal (S_(φ)) with        respective phase-shifted reference signals (S_(Q), S _(Q) ,        S_(I), S_(Ī)), the correlation signals (S_(φ,Q), S_(φ,Q) ,        S_(φ,I), S_(φ,Ī)) comprising a first set of correlation signals        (S_(φ,Q), S_(φ,Q) ) and a second set of correlation signals        (S_(φ,I), S_(φ,Ī)),    -   wherein the average phase of the first set of correlation        signals (S_(φ,Q), S_(φ,Q) ) and the average phase of the second        set of correlation signals (S_(φ,I), S_(φ,Ī)) are orthogonal or        substantially orthogonal to each other, and    -   wherein the mean time of acquisition of the first set of        correlation signals (S_(φ,Q), S_(φ,Q) ) and the mean time of        acquisition of the second set of correlation signals (S_(φ,I),        S_(φ,Ī)) are equal or substantially equal.

1. An apparatus comprising circuitry configured to: acquire correlationsignals by correlating a photodetected signal with respectivephase-shifted reference signals, the correlation signals comprisingquadrature correlation signals and in-phase correlation signals, whereinthe circuitry is configured in such a way that the mean time ofacquisition of the quadrature correlation signals and the mean time ofacquisition of the in-phase correlation signals are equal orsubstantially equal.
 2. The apparatus of claim 1, wherein the processoris configured to acquire the correlation signals according to any one ofthe phase-shift sequences: Phase-shift sequence A B C D E F G H S_(φ,Q)S_(φ,Q) S_(φ,) _(Q) S_(φ,) _(Q) S_(φ,I) S_(φ,I) S_(φ,) _(I) S_(φ,) _(I)S_(φ,I) S_(φ,) _(I) S_(φ,I) S_(φ,) _(I) S_(φ,Q) S_(φ,) _(Q) S_(φ,Q)S_(φ,) _(Q) S_(φ,) _(I) S_(φ,I) S_(φ,) _(I) S_(φ,I) S_(φ,) _(Q) S_(φ,Q)S_(φ,) _(Q) S_(φ,Q) S_(φ,) _(Q) S_(φ,) _(Q) S_(φ,Q) S_(φ,Q) S_(φ,) _(I)S_(φ,) _(I) S_(φ,I) S_(φ,I)


3. The apparatus of claim 1, wherein the reference signals arephase-shifted by 0°, 90°, 270° and 180° respectively, compared to amodulated light signal.
 4. The apparatus of claim 1, wherein thecorrelation signals are quadrature modulation signals and the circuitryis configured to acquire a first and a last correlation signal on 180°opposing phase, and a second and third correlation signal also on 180°opposing phase.
 5. The apparatus of claim 1, wherein the circuitry isconfigured to first acquire subsequently a first set of 0° and then of90° correlation measurements, and then acquire subsequently a second setof 210° and then of 120° correlation measurements.
 6. The apparatus ofclaim 3, wherein the circuitry is configured to: acquire the 0°correlation signal at a first time T₀, acquire the 90° correlationsignal at a second time T₀+ΔT, acquire the 270° correlation signal at athird time T₀+2·ΔT, and acquire the 180° correlation signal at a lasttime T₀+3·ΔT, where T₀ is a time when the acquisition of the correlationsignals starts, and where ΔT is a predefined time interval.
 7. Theapparatus of claim 1, wherein the processor is configured to acquire thecorrelation signals according to any one of the phase-shift sequences:Phase-shift sequence Picture A B C D E F G H 0  0°  0° 180° 180°  90° 90° 270° 270° 1  90° 270°  90° 270°  0° 180°  0° 180° 2 270°  90° 270° 90° 180°  0° 180°  0° 3 180° 180°  0°  0° 270° 270°  90°  90°


8. The apparatus of claim 1, wherein the processor is configured toacquire the correlation signals according to any one of the phase-shiftsequences: Phase-shift sequence Picture I J K L M N O P 0  0°  0° 120°120°  90°  90° 210° 210° 1  90° 210°  90° 210°  0° 120°  0° 120° 2 210° 90° 210°  90° 120°  0° 120°  0° 3 120° 120°  0°  0° 210° 210°  90°  90°


9. The apparatus of claim 1, further comprising an illumination unitconfigured to illuminate a scene with a modulated light signal.
 10. Theapparatus of claim 9, further comprising an imaging sensor configured toreceive the photodetected signal.
 11. An apparatus comprising circuitryconfigured to: acquire correlation signals by correlating aphotodetected signal with respective phase-shifted reference signals,the correlation signals comprising a first set of correlation signalsand a second set of correlation signals, wherein the circuitry isconfigured in such a way that the average phase of the first set ofcorrelation signals and the average phase of the second set ofcorrelation signals are orthogonal or substantially orthogonal to eachother, and wherein the circuitry is configured in such a way that themean time of acquisition of the first set of correlation signals and themean time of acquisition of the second set of correlation signals areequal or substantially equal.
 12. The apparatus of claim 11, wherein thefirst set of correlation signals comprises quadrature modulation signalsand the second set of correlation signal comprises in-phase correlationsignals.
 13. The apparatus of claim 11, wherein the first set ofcorrelation signals comprises quadrature modulation signals that arephase-shifted by a predefined phase angle and the second set ofcorrelation signals comprises in-phase correlation signals that arephase-shifted by the predefined phase angle.
 14. The apparatus of claim11, wherein the processor is configured to acquire the first set ofcorrelation signals and the second set of correlation signals accordingto any one of the phase-shift sequences: Phase-shift sequence Picture AB C D E F G H 0  0°  0° 180° 180°  90°  90° 270° 270° 1  90° 270°  90°270°  0° 180°  0° 180° 2 270°  90° 270°  90° 180°  0° 180°  0° 3 180°180°  0°  0° 270° 270°  90°  90°


15. The apparatus of claim 11, wherein the processor is configured toacquire the first set of correlation signals and the second set ofcorrelation signals according to any one of the phase-shift sequences:Phase-shift sequence Picture I J K L M N O P 0  0°  0° 120° 120°  90° 90° 210° 210° 1  90° 210°  90° 210°  0° 120°  0° 120° 2 210°  90° 210° 90° 120°  0° 120°  0° 3 120° 120°  0°  0° 210° 210°  90°  90°


16. A method comprising: acquiring correlation signals by correlating aphotodetected signal with respective phase-shifted reference signals,the correlation signals comprising quadrature correlation signals andin-phase correlation signals, wherein the acquiring the correlationsignals is performed in such a way that the mean time of acquisition ofthe quadrature correlation signals and the mean time of acquisition ofthe in-phase correlation signals are substantially equal.
 17. A computerprogram comprising instructions which, when carried out on a processor,cause the processor to perform the method of claim
 16. 18. A methodcomprising: acquiring correlation signals by correlating a photodetectedsignal with respective phase-shifted reference signals, the correlationsignals comprising a first set of correlation signals and a second setof correlation signals, wherein the average phase of the first set ofcorrelation signals and the average phase of the second set ofcorrelation signals are orthogonal or substantially orthogonal to eachother, and wherein the mean time of acquisition of the first set ofcorrelation signals and the mean time of acquisition of the second setof correlation signals are equal or substantially equal.
 19. A computerprogram comprising instructions which, when carried out on a processor,cause the processor to perform the method of claim 18.